Rearview vision system for vehicle including panoramic view

ABSTRACT

A rearview vision system for a vehicle includes at least one image capture device directed rearwardly with respect to the direction of travel of the vehicle. A display system displays an image synthesized from output of the image captive device. The display system is preferably contiguous with the forward field of view of the vehicle driver at a focal length that is forward of the vehicle passenger compartment. A plurality of image capture devices my be provided and the display system displays a unitary image synthesized from outputs of the image captive devices which approximates a rearward-facing view from a single location, such as forward of the vehicle.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of patent application Ser.No. 08/023,918 filed Feb. 26, 1993, now U.S. Pat. No. 5,550,677, byKenneth Schofield and Mark Larson.

BACKGROUND OF THE INVENTION

This invention relates generally to vision systems for vehicles and,more particularly, to rearview vision systems which provide the vehicleoperator with scenic information in the direction rearward of thevehicle. More particularly, the invention relates to a rearview visionsystem utilizing image capture devices, such as CMOS imaging arrays andthe like.

A long-felt need in the art of vehicle rearview vision systems has beento eliminate exterior rearview mirrors by utilizing image capturedevices, such as cameras, in combination with dashboard displays. Thiswould be beneficial because it would reduce wind drag on the vehicle,wind noise and vehicle weight. Furthermore, rearview mirrors protrude asubstantial distance from the side of the vehicle, which makesmaneuvering in tight spaces more difficult. Image capture devices arecapable of positioning in a greater variety of locations on the vehicle,providing more flexibility of vehicle styling. It is further expectedthat camera systems would greatly reduce the blind spots to the sidesand rear of the vehicle common with vehicles equipped with conventionalrearview mirror systems. The driver cannot perceive vehicles, objects,or other road users in such blind spots without turning his or her body,which interferes with forward-looking visual activities.

Camera-based rearview vision systems for vehicles have not obtainedcommercial acceptance. One difficulty with proposed systems has beenthat they present a large amount of visual information in a manner whichis difficult to comprehend. This difficulty arises from many factors. Inorder to significantly reduce blind spots, multiple image capturedevices are typically positioned at various locations on the vehicle.The image of an object behind the equipped vehicle is usually capturedby more than one image capture device at a time and displayed inmultiple images. This may confuse the driver as to whether more than oneobject is present. When multiple image capture devices are positioned atdifferent longitudinal locations on the vehicle, objects behind thevehicle are at different distances from the image capture devices. Thisresults in different image sizes for the same object. This effect isespecially noticeable for laterally extending images, such as a bridge,highway crosswalk markings, the earth's horizon, and the like. Suchimages are at different vertical angles with respect to the imagecapture devices. This results in different vertical positions on thedisplay causing the elongated image to appear disjointed.

A camera system provides a monocular view of the scene, compared to thebinocular, or stereoscopic, view obtained when the scene is viewedthrough a rearview mirror. This makes the ability to judge distances ina camera system a problem. This effect is most noticeable at distancesclose to the vehicle where stereoscopic imaging is relied uponextensively by the driver in judging relative locations of objects.Therefore, known camera systems fail to provide to the driver importantinformation where that information is most needed--at small separationdistances from surrounding objects.

Another difficulty with camera systems is that, in order to provide asufficient amount of information, the camera system typically presentsthe driver with a greatly increased field of view. This improvesperformance by further reducing blind spots at the side and rear of thevehicle. However, an increased field of view is often obtained byutilizing a wide-angle lens which introduces distortion of the scene andfurther impairs the ability of the driver to judge distances of objectsdisplayed. The problem with such distortion of the scene is that thedriver must concentrate more on the display and take a longer time tointerpret and extract the necessary information. This further distractsthe driver from the primary visual task of maintaining awareness ofvehicles and other objects in the vicinity of the driven vehicle.

SUMMARY OF THE INVENTION

The present invention is directed towards enhancing the interpretationof visual information in a rearview vision system by presentinginformation in a manner which does not require significant concentrationof the driver or present distractions to the driver. This isaccomplished according to the invention in a rearview vision systemhaving at least two image capture devices positioned on the vehicle anddirected rearwardly with respect to the direction of travel of thevehicle. A display is provided for images captured by the image capturedevices. The display combines the captured images into an image thatwould be achieved by a single rearward-looking camera having a viewunobstructed by the vehicle. In order to obtain all of the necessaryinformation of activity, not only behind but also along side of thevehicle, the virtual camera should be positioned forward of the driver.The image synthesized from the multiple image capture devices may have adead space which corresponds with the area occupied by the vehicle. Thisdead space is useable by the driver's sense of perspective in judgingthe location of vehicles behind and along side of the equipped vehicle.

The present invention provides techniques for synthesizing imagescaptured by individual, spatially separated, image capture devices intosuch ideal image, displayed on the display device. This may beaccomplished according to an aspect of the invention by providing atleast three image capture devices. At least two of the image capturedevices are side image capture devices mounted on opposite sides of thevehicle. At least one of the image capture devices is a center imagecapture device mounted laterally between the side image capture devices.A display system displays an image synthesized from outputs of the imagecapture devices. The displayed image includes an image portion from eachof the image capture devices. The image portion from the center imagecapture device is vertically compressed.

It has been discovered that such vertical compression substantiallyeliminates distortion resulting from the spatial separation between thecameras and can be readily accomplished. In an illustrated embodiment,the image compression is carried out by removing selective ones of thescan lines making up the image portion. A greater number of lines areremoved further away from the vertical center of the image.

The compression of the central image portion produces a dead space inthe displayed image which may be made to correspond with the area thatwould be occupied by the vehicle in the view from the single virtualcamera. Preferably, perspective lines are included at lateral edges ofthe dead space which are aligned with the direction of travel of thevehicle and, therefore, appear in parallel with lane markings. Thisprovides visual clues to the driver's sense of perspective in order toassist in judging distances of objects around the vehicle.

According to another aspect of the invention, image enhancement meansare provided for enhancing the displayed image. Such means may be in theform of graphic overlays superimposed on the displayed image. Suchgraphic overlap may include indicia of the anticipated path of travel ofthe vehicle which is useful in assisting the driver in guiding thevehicle in reverse directions. Such graphic overlay may include adistance grid indicating distances behind the vehicle of objectsjuxtaposed with the grid.

These and other objects, advantages, and features of this invention willbecome apparent by review of the following specification in conjunctionwith the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 a top plan view of a vehicle having a rearview vision systemaccording to the invention;

FIG. 2 is a side elevation of the vehicle in FIG. 1;

FIG. 3 is a front elevation of a display according to the invention;

FIG. 4 the same view as FIG. 1 illustrating an alternative embodiment ofthe invention;

FIG. 5 is a block diagram of an electronic system according to theinvention;

FIG. 6 is the same view as FIG. 3 illustrating an alternate mode ofoperation of the system;

FIG. 7 is the same view as FIG. 2 illustrating an alternative embodimentof the invention;

FIG. 8 is the same view as FIG. 3 illustrating an alternative embodimentof the invention;

FIG. 9 is the same view as FIGS. 1 and 4 illustrating an alternativeembodiment of the invention;

FIG. 10 is the same view as FIGS. 3 and 8 illustrating an alternativeembodiment of the invention; and

FIG. 11 is a chart illustrating the horizontal row of pixels (n1, n2) onwhich an object will be imaged from two longitudinally separated imagecapture devices as that object is spaced at different longitudinaldistances from the image capture devices.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring now specifically to the drawings, and the illustrativeembodiments depicted therein, a vehicle 10, which may be an automobile,a light truck, a sport utility vehicle, a van, a bus, a large truck, orthe like includes a rearview vision system, generally illustrated at 12,for providing a driver of the vehicle with a view rearwardly of thevehicle with respect to the direction of travel D of the vehicle (FIG.1). Vision system 12 includes at least two side image capture devices 14positioned, respectively, on opposite sides of vehicle 10 and a centerimage capture device 16 positioned on the lateral centerline of thevehicle. All of the image capture devices are directed generallyrearwardly of the vehicle. Rearview vision system 12 additionallyincludes an image processor 18 for receiving data signals from imagecapture devices 14, 16 and synthesizing, from the data signals, acomposite image 42 which is displayed on a display 20.

As will be set forth in more detail below, the images captured by imagecapture devices 14, 16 are juxtaposed on display 20 by image processor18 in a manner which approximates the view from a single virtual imagecapture device positioned forwardly of the vehicle at a location C andfacing rearwardly of the vehicle, with the vehicle being transparent tothe view of the virtual image capture device. Vision system 12 providesa substantially seamless panoramic view rearwardly of the vehiclewithout duplicate or redundant images of objects. Furthermore,elongated, laterally-extending, objects, such as the earth's horizon,appear uniform and straight across the entire displayed image. Thedisplayed image provides a sense of perspective, which enhances theability of the driver to judge location and speed of adjacent trailingvehicles.

Each of side image capture devices 14 has a field of view 22 and isaimed rearwardly with respect to the vehicle about an axis 24 which isat an angle, with respect to the vehicle, that is half of the horizontalfield of view of the image capture device. In this manner, each of theimage capture devices 14 covers an area bounded by the side of thevehicle and extending outwardly at an angle defined by the horizontalfield of view of the respective side image capture device. Center imagecapture device 16 has a horizontal field of view 26, which issymmetrical about the longitudinal axis of the vehicle. The field ofview of each side image capture device 14 intersect the field of view ofcenter image capture device 16 at a point P which is located a distanceQ behind vehicle 10.

Rear blind zones 30 are located symmetrically behind vehicle 10extending from the rear of the vehicle to point P. Side blind zones 25located laterally on respective sides of the vehicle extend rearwardlyof the forward field of view 36 of the driver to the field of view 22 ofthe respective side image capture device 14. An object will not becaptured by side image capture devices 14 or center image capturedevices 16 if the object is entirely within one of the blind zones 25,30. In order for an object, such as another vehicle V or other road usertravelling to the side of vehicle 10, to be observed by an operator ofvehicle 10, the object must be either at least partially within theforward field of view 36 of the driver or be captured by image capturedevices 14, 16 and displayed on display 20. FIG. 4 illustrates vehicle10 travelling on a three-lane highway having lanes L1, L2, and L3 withthe vehicle in lane L2. Another vehicle V is shown positioned mostlywithin one of the blind zones 25, but with the rearmost portion of thevehicle V extending into field of view 22 where the vehicle image willbe captured by one of side image capture devices 14. In the illustratedembodiment, vehicle V is a motorcycle travelling in the center of lanesL1 or L3 and represents a worst case for observing a vehicle travellingat least partially within one of the blind zones 25. In order for aportion of vehicle V to be extending either forwardly or rearwardly ofthe respective blind zone 25, where the vehicle V may be observed byeither the forward field of view 36 of the driver or by the rearviewvision system 12, the field of view 22 of side image capture devices 14must be sufficiently wide to capture a portion of vehicle V asillustrated in FIG. 4. Preferably, the horizontal field of view 22 ofside image capture devices 14 is no greater than that required toprovide sufficient coverage which would be in the range of betweenapproximately 55 degrees and approximately 70 degrees. In theillustrated embodiment, the horizontal field of view 22 is 61 degrees.In order for a portion of vehicle V to be within a vertical field ofview 40 of one of side image capture devices 14, the field of viewshould extend to the pavement at a plane M which intersects vehicle V(FIG. 2). Preferably, vertical field of view 40 is between approximately60 degrees and approximately 75 degrees. In the illustrated embodiment,vertical field of view 40 is 66 degrees.

A left overlap zone 32 and a right overlap zone 34 extend rearward fromrespective points P where the horizontal fields of view of the sideimage capture devices intersect the field of view of center imagecapture device 16. Overlap zones 32, 34 define areas within which anobject will be captured both by center image capture device 16 and oneof the side image capture devices 14. An object in an overlap zone 32,34 will appear on display 20 in multiple image portions in a redundantor duplicative fashion. In order to avoid the presentation of redundantinformation to the driver, and thereby avoid confusion and simplify thetask of extracting information from the multiple images or combinedimages on display 20, the object should avoid overlapping zones 32, 34.In practice, this may be accomplished to a satisfactory extent by movingpoints P away from the vehicle and thereby increasing distance Q. It isdesirable to increase distance Q to a length that will exclude vehiclestravelling at a typical separation distance behind vehicle 10 fromoverlapping zones 32, 34. This separation distance is usually a functionof the speed at which the vehicles on the highway are travelling. Thefaster the vehicles are travelling, the further Q should be moved behindvehicle 10 to keep overlap zones 32 and 34 outside of the recommendedvehicle spacing. If, however, the vehicles are travelling at a slowerspeed, then the generally accepted recommendation for vehicle spacingdecreases and it is more likely that a vehicle will be within overlapzone 32, 34. Therefore, the distance Q may be selected to accommodateexpected vehicle spacing for an average driving speed of vehicle 10.

Distance Q is a function Of the effective horizontal field of view 26 ofcenter image capture device 16. As field of view 26 decreases, points Pmove further rearward of the vehicle from a distance Q₁, to a distanceQ₂, as best illustrated in FIG. 4. In order to increase distance Q toeliminate redundant and duplicative information displayed on display 20for most driving conditions of vehicle 10, field of view 26 ispreferably less than 12 degrees. In the illustrated embodiment, field ofview 26 is between 6 and 8 degrees. Alternatively, distance Q may bedynamically adjusted according to some parameter, such as the speed ofvehicle 10. This would allow Q to be greater when the vehicle istravelling at a faster speed, where vehicle separation tends to belarger, and vice versa. Field of view 26 may be adjusted by utilizing aselective presentation of pixels of the captured image in the displayedimage.

Referring to FIG. 3, image display device 20 displays a composite image42 made up of a left image portion 44, a right image portion 46, and acenter image portion 48. Each image portion 44-48 is reversed from theimage as captured by the respective image capture device 14, 16utilizing conventional techniques. These techniques include reading theimage in reverse with the image capture device, writing the image inreverse to display device 20, or reversing the image in image processor18. Left image portion 44 is joined with central image portion 48 at aboundary 50. Central image portion 48 is joined with right image portion46 at a boundary 52. As may best be seen in FIG. 3, the image portionsat boundaries 50 and 52 are continuous whereby composite image 42 is aseamless panoramic view rearwardly of the vehicle. As also is apparentfrom FIG. 3, central image portion 48 is narrower than either left imageportion 44 or right image portion 46. This is a result of reducing thehorizontal field of view 26 of center image capture device 16sufficiently to move points P, and thus overlap zones 32 and 34, asufficient distance behind vehicle 10 to reduce redundant andduplicative images between image portions 44-48. Composite image 42provides a clear image, which avoids confusion and simplifies the taskof extracting information from the multiple image portions 44-48. Asalso may be seen by reference to FIG. 3, display 20 may additionallyinclude indicia such as the readout of a compass 54, vehicle speed 56,turn signals 58, and the like as well as other graphical or videodisplays, such as a navigation display, a map display, and aforward-facing vision system. In this manner, rearview vision system 12may be a compass vision system or an information vision system.

In the embodiment of rearview vision system 12 having a dynamicallyadjusted value of distance Q, the spacing between boundaries 50 and 52will dynamically adjust in sequence with the adjustment of distance Q.Thus, as overlap zones 32, 34 move further away from the vehicle; forexample, in response to an increase in speed of the vehicle, boundarylines 50 and 52 will move closer together and vice versa. In thismanner, composite image 42 is dynamic, having image portions ofdynamically adaptive sizes.

Display 20 is of a size to be as natural as possible to the driver. Thisis a function of the size of the display and the distance between thedisplay and the driver. Preferably, the displayed image simulates animage reflected by a rearview mirror. As such, the size of display 20 isapproximately the combined areas of the three rearview mirrors (oneinterior mirror and two exterior mirrors) conventionally used withvehicles. As best seen by reference to FIG. 2, display 20 is preferablypositioned within the driver's physiological field of view withoutobstructing the view through the windshield. It is known that thedriver's field of view, with the head and eyes fixed forward, extendsfurther in a downward direction than in an upward direction. Display 20could be located above the vertical view through the windshield whereinthe display may be observed at the upward portion of the driver's fieldof view. However, the position for the display illustrated in FIG. 2 ispreferred wherein the display is within the lower portion of thedriver's field of view.

Display 20, in the illustrated embodiment, is a flat panel display, suchas a back-lit liquid crystal display, a plasma display, a field emissiondisplay, or a cathode ray tube. However, the synthesized image could bedisplayed using other display techniques such as to provide a projectedor virtual image. One such virtual display is a heads-up display. Thedisplay may be mounted/attached to the dashboard, facia or header, or tothe windshield at a position conventionally occupied by an interiorrearview mirror.

Although various camera devices may be utilized for image capturedevices 14, 16, an electro-optic, pixelated imaging array, located inthe focal plane of an optical system, is preferred. Such imaging arrayallows the number of pixels to be selected to meet the requirements ofrearview vision system 12. The pixel requirements are related to theimaging aspect ratio of the respective image capture devices, which, inturn, are a function of the ratio of the vertical-to-horizontal field ofview of the devices, as is well known in the art. In the illustratedembodiment, the imaging aspect ratio of side image capture devices 14 is2:1 and the image aspect ratio of central image capture device 16 isvariable down to 0.1:1. Such aspect ratio will produce images which willnot typically match that of commercially available displays. Acommercially available display may be used, however, by leaving ahorizontal band of the display for displaying alpha-numeric data, suchas portions of an instrument cluster, compass display, or the like, asillustrated in FIG. 3.

In the illustrated embodiment, image capture devices 14, 16 are CMOSimaging arrays of the type manufactured by VLSI Vision Ltd. ofEdinburgh, Scotland, which are described in more detail in co-pendingU.S. patent application Ser. No. 08/023,918 filed Feb. 26, 1993, byKenneth Schofield and Mark Larson for an AUTOMATIC REARVIEW MIRRORSYSTEM USING A PHOTOSENSOR ARRAY, the disclosure of which is herebyincorporated herein by reference. However, other pixelated focal planeimage-array devices, which are sensitive to visible or invisibleelectromagnetic radiation, could be used. The devices could be sensitiveto either color or monochromatic visible radiation or near or farinfrared radiation of the type used in night-vision systems. Each imagecapture device could be a combination of different types of devices,such as one sensitive to visible radiation combined with one sensitiveto infrared radiation. Examples of other devices known in the artinclude charge couple devices and the like.

Preferably, image capture devices 14 and 16 are all mounted at the samevertical height on vehicle 10, although compromise may be required inorder to accommodate styling features of the vehicle. The horizontal aimof image capture devices 14 and 16 is preferably horizontal. However,the portion of the image displayed is preferably biased toward thedownward portion of the captured image because significantly less usefulinformation is obtained above the horizontal position of the imagecapture devices.

Each image-capturing device 14, 16 is controlled by appropriatesupporting electronics (not shown) located in the vicinity of theimaging array such that, when operating power is supplied, either ananalog or a digital data stream is generated on an output signal linesupplied to image processor 18. The support electronics may be providedpartially on the image chip and partially on associated electronicdevices. For each exposure period, a value indicative of the quantity oflight incident on each pixel of the imaging array during the exposureperiod is sequentially outputted in a predetermined sequence, typicallyrow-by-row. The sequence may conform to video signal standards whichsupport a direct view such that, when a scene is viewed by animage-capturing device, the image presented on a display representsdirectly the scene viewed by the image-capturing devices. However, whenlooking forward and observing a displayed image of a rearward scene, thedriver will interpret the image as if it were a reflection of the sceneas viewed through a mirror. Objects to the left and rearward of thevehicle, as viewed by the rearward-looking camera, are presented on theleft-hand side of the display and vice versa. If this reversal iseffected in image processor 18, it may be by the use of a data storagedevice, or buffer, capable of storing all of the pixel values from oneexposure period. The data is read out of the data storage device in areversed row sequence. Alternatively, the imaging array electronicscould be constructed to provide the above-described reversal at theimage-capturing device or at the display.

Data transmission between image capture devices 14, 16 and imageprocessor 18 and/or between image processor 18 and display 20 may be byelectrically conductive leads or fiber-optic cable. It is possible, forparticular applications, to eliminate image processor 18 and directdrive display 20 from image capture devices 14, 16 at the pixel level.

The data streams from image-capturing devices 14, 16 are combined inimage processor 18 and directly mapped to the pixel array of display 20.This process is repeated preferably at a rate of at least 30 times persecond in order to present an essentially real time video image. Theimage captured by side image capture device 14 on the right side of thevehicle is presented in right image portion 46 and the image from sideimage capture device 14 on the left side of the vehicle is displayed onleft image portion 44. The image from center image capture device 16 isdisplayed on central image portion 48. The three image portions 44-48are presented in horizontal alignment and adjacent to each other.However, the composite image may be positioned at any desired verticalposition in the display 20. It is also possible to display imageportions 44-48 on separate image devices which are adjacent each other.

In vision system 12, side image capture devices 14 are positionedpreferably at a forward longitudinal position on vehicle 10 and centerimage capture device 16 is positioned at a rearward longitudinalposition on the vehicle. As best seen by reference to FIG. 7, thispositioning creates a difference in the vertical angle between each sideimage capture device 14 and center image capture device 16 with respectto a fixed location P₁ that is a distance D₁ behind the vehicle. Thisdifference in sensing angle will cause each side image capture device 14to image an object located at P₁ on a horizontal row of pixels that isdifferent from the horizontal row of pixels that center image capturedevice 16 will image the same object. If the image is below thehorizontal centerline of the image capture device, it will be imaged ona lower row of pixels by center image capture device 16 than the row ofpixels it will be imaged by the side image capture devices 14, asillustrated in FIG. 7. This mismatch between horizontal pixel rows ofthe captured image is furthermore a function of the distance of thecaptured image from the rear of the vehicle. This can be understood byreference to FIG. 11 which presents a chart 90 having a first column 92of pixel lines n1, measured from the array centerline, at which anobject will be imaged by side image capture device 14 and a secondcolumn 94 of pixel lines n2, measured from the array verticalcenterline, at which the same object will be imaged by center imagecapture device 16. The result is that an object, which is captured byboth side and center image capture devices 14, 16, will be verticallydisjointed at the boundary of the displayed image, if the object iscaptured by more than one image capture device. The amount ofdisjointment will be greater closer to the vehicle and less at furtherdistances. If the object is elongated in the horizontal direction, suchas earth's horizon, bridges, or cross-markings on highways, then theobject will appear to be either broken or crooked.

In order to provide uniform display of laterally elongated images, arearview vision system 12' is provided having a central image portion48' which is processed differently from the image display portions 44'and 46' produced by the side image capture devices (FIG. 8). Centralimage portion 48' is reduced vertically, or compressed, by removingspecified scan lines, or pixel rows, from the image captured by centerimage capture device 16 in a graduated fashion. The difference in thepixel line at which an object will be imaged by each of the side andcenter image capture devices is a function of the distance D of theobject from the rear of the vehicle, with a greater variation occurringat shorter distances and the variation reducing to zero for infinitedistances. Therefore, the compression of central image portion 48' isnon-linear, with substantially no compression at the vertical center ofthe image and greater compression at greater distances above and belowthe vertical center point of the image. This is accomplished by removingspecific lines from the center display in a graduated fashion with agreater number of lines removed further from the vertical center of theimage. The removed lines may be merely discarded in order to verticallyreduce the image. Alternatively, the data contained in the removed linesmay be utilized to modify the value of adjacent pixels above and belowthe removed line in order to enhance the quality of the compressedimage. Averaging, median filtering, or other such known techniques mayalso be used.

Each of right image portion 46' and left image portion 44' includes anupper portion 64 which extends above the compressed upper portion of thecentral image portion 48'. In the illustrated embodiment, upper portions64 are deleted in order to present a uniform upper horizontal boundaryfor display 20'. In the illustrated embodiment, the mismatch between thelower horizontal boundary of central image portion 48' and each of theleft and right image portions provides a dead space 66 which provides avisual prompt to the user of the approximate location of the rearwardcorners S of vehicle 10. This dead space 66 in the image displayed ondisplay 20' approximates the footprint occupied by vehicle 10 whenviewed from point C. This is particularly useful because it provides avisual indication to the driver that a vehicle passing vehicle 10, asviewed in either left image portion 44' or right image portion 46', isat least partially adjacent vehicle 10 if the image of the approachingvehicle is partially adjacent to dead space 66.

In an alternative embodiment, the vertical compression technique may beapplied to only a lower vertical portion of central image portion 48'.In most driving situations, objects imaged by rearward-facing imagecapture devices above the horizon are at a long distance from thevehicle while those below the horizon get progressively closer to thevehicle in relation to the distance below the horizon in the displayedimage. Therefore, compression of the upper vertical portion of thecentral image portion may be eliminated without significant reduction inperformance.

Compression of the central image portion may also advantageously beprovided horizontally, as well as vertically. Spatial separation ofcenter image capture device 16 from side image capture devices 14 causessimilar distortion, as that described above, in the horizontaldirection. This effect is spherical in nature and would require a morecomplex corrective action, such as compressing the image based upon theremoval of pixels from an approximation to concentric circles centeredon the center of the imaging array, or other techniques which would beapparent to those skilled in the art.

A rearview vision system 12" includes an image display 20" having acompressed central image portion 48" and left and right image portions44" and 46", respectively (FIG. 10). A border 50' between left sideimage 44" and central image 48" includes a vertical central borderportion 50a', an upper border portion 50b', and a lower border portion50c'. Upper border portion 50b' and lower border portion 50c' divergelaterally outwardly, vertically away from central portion 50a'. A border52' between central image portion 48" and right image portion 46"includes a central boundary portion 52a', an upper boundary portion52b', and a lower boundary portion 52c'. Upper boundary portion 52b' andlower boundary portion 52e' diverge laterally outwardly vertically awayfrom central portion 52a'. This creates an upper portion of centralimage portion 48" and a lower portion of central image portion 48" whichextend beyond the center portion thereof. This configuration is basedupon the realization that the surface of the road immediately behind thevehicle is captured by central image capture device 16. Likewise, thehorizontal plane above the vehicle, which is symmetrical with the roadsurface, is captured by the center image capture device. This may beseen by referring to point P in FIG. 10, which illustrate the pointswhere the effective radius 68 of the virtual image capture deviceintersects dead zones 30 and by referring to point S in FIG. 10 whichillustrates the corners or the rear of the vehicle (S).

The image displayed on display 20" includes a dead space 66' havingdiverging lateral sides 68a, 68b. Diverging sides 68a and 68b areconfigured in order to extend in the direction of travel of vehicle 10which is parallel to lane markings of a highway on which vehicle 10 istravelling. This further enhances the visual perception of the driver byproviding a visual clue of the location of images appearing on display20" with respect to the vehicle 10. Side portions 68a, 68b, in theillustrated embodiment, are natural extensions of lower boundaryportions 50c' and 52c' and extend from point S on each respective sideof the vehicle to point R, which represents the intersection of thelower extent of the vertical field of view 40 of each side image capturedevice 14 with the pavement (FIG. 7).

Rearview vision systems 12' and 12" utilize a displayed synthesizedimage which takes into account the use of perspective in enhancing thedriver's understanding of what is occurring in the area surrounding thevehicle. The images produced on displays 20' and 20" effectively removethe vehicle bodywork and replace the bodywork with a vehicle footprintas would be viewed by virtual camera C. The image displayed on display20" further includes perspective lines which further enhance the roll ofperspective in the driver's understanding of what is occurring.

In order to further enhance the driver's understanding of what isoccurring in the area surrounding the vehicle, a rearview vision system12'" includes a display 20'" having image enhancements (FIG. 6). In theillustrative embodiment, such image enhancements include graphicoverlays 70a, 70b which are hash marks intended to illustrate to thedriver the anticipated path of movement of vehicle 10. In theillustrated embodiment, the anticipated vehicle motion is a function ofthe vehicle direction of travel as well as the rate of turn of thevehicle. The forward or rearward direction of vehicle travel isdetermined in response to the operator placing the gear selection device(not shown) in the reverse gear position. The degree of turn of thevehicle may be determined by monitoring the movement of the vehiclesteering system, monitoring the output of an electronic compass, ormonitoring the vehicle differential drive system. In the embodimentillustrated in FIG. 6, the configuration of graphic overlays 70a, 70bindicates that the vehicle is in reverse gear and that the wheels areturned in a manner that will cause the vehicle to travel toward thedriver's side of the vehicle. If the wheels were turned in the oppositedirection, graphic overlays 70a, 70b would curve clockwise toward theright as viewed in FIG. 6. If the vehicle's wheels were straight,graphic overlays 70a, 70b would be substantially straight converginglines. If the vehicle is not in reverse gear position, graphic overlays70a, 70b are not presented. Other types of graphic overlays of thedisplayed image are comprehended by the invention.

Horizontal grid markings on the display may be provided to indicatedistances behind the vehicle at particular markings. Such grid wouldallow the driver to judge the relative position of vehicles behind theequipped vehicle. In one embodiment, short horizontal lines aresuperimposed on the displayed image at regular rearward intervals inhorizontal positions which correspond to the boundaries of the lane inwhich the vehicle is travelling. In order to avoid confusion when thevehicle is travelling in a curved path, from a lack of correspondencebetween the graphic overlay and the road, a signal indicative of thevehicle's rate of turn may be taken into account when generating thegraphic overlay. In this manner, the distance indications may be movedlaterally, with reduced horizontal separation, to correspond to thepositions of the curved lane boundaries and vertically on the image tocompensate for the difference between distances along a straight andcurved path.

Another image enhancement is to alter the appearance of an object in aparticular zone surrounding the vehicle in order to provide anindication, such as a warning, to the driver. As an example, a vehiclethat is too close to the equipped vehicle for safe-lane change, may bedisplayed in a particular color, such as red, may flash, or otherwise bedistinguishable from other images on the display. Preferably, the speedof the equipped vehicle 10, which may be obtained from known speedtransducers, may be provided as an input to the rearview vision systemin order to cause such warning to be a function of the vehicle speedwhich, in turn, affects the safe separation distance of vehicles. Theoperation of the turn signal may also be used to activate suchhighlighting of other road users or to modify the scope of the imagedisplayed. In order to determine the distance of objects behind vehicle10, a separate distance-measuring system may be used. Such separatesystem may include radar, ultrasonic sensing, infrared detection, andother known distance-measuring systems. Alternatively, stereoscopicdistance-sensing capabilities of side image capture devices 14 may beutilized to determine the separation distance from trailing objectsutilizing known techniques.

Thus, it is seen that the image displayed on display 20-20'" may bedifferent under different circumstances. Such different circumstancesmay relate to the vehicle's direction of travel, speed, rate of turn,separation from adjacent objects, and the like.

Various other forms of image processing may be utilized with rearviewvision system 12-12'". Luminant and chrominant blending may be appliedto the images captured by image capture devices 14, 16 in order toproduce equality of the image data whereby the image port order toproduce equality of the image data whereby the image portions appear asif they were produced by one image capture device. The dynamic range ofthe image capture devices may be extended in order to provide highquality images under all lighting conditions. Furthermore, individualpixel groups may be controlled in order to selectively compensate forbright or dark spots. For example, anti-blooming techniques may beapplied for bright spots. Multiple exposure techniques may be applied tohighlight dark areas. Image morphing and warping compensation techniquesmay additionally be applied. Resolution of the image capture devices anddisplay may be selected in order to provide sufficient image quality forthe particular application.

A heater may be applied to each image capture device in order to removedew and frost that may collect on the optics of the device. Although, inthe illustrative embodiment, the optical centerline of the cameracoincides with the field of view, particular applications may result inthe centerline of the camera pointing in a direction other than thecenterline of the field of view. Although, in the illustrativeembodiment, the image capture devices are fixed, it may be desirable toprovide selective adjustability to the image capture devices or opticalpaths in particular applications. This is particularly desirable whenthe system is used on articulated vehicles where automated andcoordinated camera aim may be utilized to maintain completeness of thesynthesized image.

When operating the vehicle in the reverse direction, it may be desirableto provide additional data concerning the area surrounding the immediaterear of the vehicle. This may be accomplished by utilizingnon-symmetrical optics for the center image capture device in order toprovide a wide angle view at a lower portion of the field of view.Alternatively, a wide angle optical system could be utilized with theelectronic system selectively correcting distortion of the capturedimage. Such system would provide a distortion-free image while obtainingmore data, particularly in the area surrounding the back of the vehicle.

The invention additionally comprehends the use of more than three imagecapture devices. In addition to side image capture devices positioned atthe front sides of the vehicle and a center image capture devicepositioned at the center rear of the vehicle, additional image capturedevices may be useful at the rear comers of the vehicle in order tofurther eliminate blind spots. It may additionally be desirable toprovide an additional center image capture device at a higher elevationin order to obtain data immediately behind the vehicle and thereby fillin the road surface detail immediately behind the vehicle. Suchadditional detail is particularly useful when operating the vehicle inthe reverse direction. Of course, each of the image capture devicescould be a combination of two or more image capture devices.

Although the present invention is illustrated as used in a rearviewvision system, it may find utility in other applications. For example,the invention may be useful for providing security surveillance in anarea where a building or other object obstructs the view of the areaunder surveillance. Additionally, the invention may find application innight-vision systems and the like. For example, the invention may beapplied to forward-facing night-vision systems, or other visionenhancement systems such as may be used in adverse weather oratmospheric conditions such as fog, applied to provide an enhanceddisplay of a synthesized image, which approximates a forward-facing viewfrom a single virtual camera located rearwardly of the driver, takingadvantage of the perspective features of the image.

Thus, it is seen that the present invention enhances the relationshipbetween the driver's primary view and the image presented on therearview vision system. This is accomplished in a manner which providesease of interpretation while avoiding confusion so that the driver doesnot have to concentrate or look closely at the image. In this manner,information presented on the display is naturally assimilated. This isaccomplished while reducing blind spots so that other vehicles orobjects of interest to the driver will likely be displayed to thedriver. Additionally, the use of perspective allows distances to be moreaccurately determined.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the invention,which is intended to be limited only by the scope of the appendedclaims, as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

The embodiments of the invention in which an exclusive property orprivilege is claimed are defined as follows:
 1. A rearview vision systemfor a vehicle, comprising:at least two image capture devices positionedon the vehicle and directed rearwardly with respect to the direction oftravel of said vehicle; and a display system which displays an imagesynthesized from outputs of said image capture devices and whichapproximates a rearward-facing substantially seamless panoramic viewfrom a single location substantially without duplicate redundant imagesof objects.
 2. The rearview vision system in claim 1 wherein said singlelocation is forward of the driver with respect to said direction oftravel and said rearward-facing view is unobstructed by the vehicle. 3.The rearview vision system in claim 1 wherein said image capture devicesare spatially separated.
 4. The rearview vision system in claim 1wherein the displayed image includes a visual prompt of the location ofsaid vehicle in said view.
 5. The rearview vision system of claim 1wherein said at least two image capture devices comprises at least threeimage capture devices.
 6. The rearview vision system in claim 1 whereinsaid display system includes multiple contiguous display surfaces, oneof said surfaces for each of said images.
 7. The rearview vision systemin claim 1 wherein said display system includes a display surface fordisplaying all of said images.
 8. The rearview vision system in claim 1wherein each of said image capture devices is a CMOS imaging array. 9.The rearview vision system in claim 4 wherein said visual prompt is andoutline of an area which would substantially be occupied by said vehiclein said view.
 10. The rearview vision system in claim 4 wherein saiddead space includes perspective lines which are aligned with saiddirection of travel.
 11. The rearview vision system in claim 5 whereinsaid at least three image capture devices include at least two sideimage capture devices positioned on opposite lateral sides of saidvehicle and at least one center image capture device laterally betweensaid side image capture devices.
 12. The rearview vision system in claim5 wherein said at least three image capture devices are at substantiallythe same height.
 13. The rearview vision system in claim 5 wherein saidat least three image capture devices are aimed along non-parallel axes.14. The rearview vision system in claim 11 wherein the displayed imageincludes an image portion from each of said image capture devices andwherein said image portion from said center image capture device has avertically central portion and vertically upper and lower portions,wherein said upper and lower portions are laterally wider than the saidcentral portion.
 15. The rearview vision system in claim 11 wherein thedisplayed image includes an image portion from each of said imagecapture devices and wherein said image portion from said center imagecapture device is compressed.
 16. The rearview vision system in claim 11wherein the displayed image includes an image portion from each of saidimage capture devices and wherein each of said image portions has ahorizontal field of view that is less than 70 degrees.
 17. The rearviewvision system in claim 11 wherein the displayed image includes an imageportion from said center image capture device which has a horizontalwidth that is narrower than the horizontal width of image portions fromsaid side image capture devices.
 18. The rearview vision system in claim16 wherein said horizontal field of view of said center image portionfrom said center capture device is less than approximately 12 degrees.19. The rearview vision system in claim 17 wherein said horizontal widthof said image portion from said center image capture device isdynamically adjustable.
 20. The rearview vision system in claim 18wherein said horizontal field of view of said image portion from saidcenter image capture device is between approximately 3 degrees andapproximately 8 degrees.
 21. The rearview vision system in claim 19wherein said horizontal width of said image portion from said centerimage capture device is adjustable in response to vehicle speed.
 22. Arearview vision system for a vehicle, comprising:at least three imagecapture devices mounted to the vehicle and directed rearwardly withrespect to the direction of travel of said vehicle, at least two of saidimage capture devices being side image capture devices respectivelymounted on opposite lateral sides of said vehicle and at least one ofsaid image capture devices being a center image capture device mountedbetween said side image capture devices wherein said side image capturedevices and said center image capture device capture objects rearwardlyof the vehicle from different angles; and a display system whichdisplays an image synthesized from outputs of said image capturedevices, the displayed image including image portions from each of saidimage capture devices combined at boundaries; wherein the image portionfrom said center image capture devices includes a portion which iscompressed thereby reducing disjointment at said boundaries.
 23. Therearview vision system in claim 22 wherein said portion of said imageportion from said center image capture device is vertically compressed.24. The rearview vision system in claim 22 wherein said image portionfrom said center image capture device has a vertically central portionand vertically upper and lower portions, wherein said upper and lowerportions are laterally wider than said central portion.
 25. The rearviewvision system in claim 22 including a visual prompt of the vehiclevertically below said image portion from said center image capturedevice.
 26. The rearview vision system in claim 22 wherein said displaysystem includes multiple contiguous display surfaces, one of saidsurfaces for each of said image portions.
 27. The rearview vision systemin claim 22 wherein said display system includes a display surface fordisplaying all of said image portions.
 28. The rearview vision system inclaim 22 wherein each of said image capture devices is a CMOS imagingarray.
 29. The rearview vision system in claim 22 wherein said at leastthree image capture devices are at substantially the same height on thevehicle.
 30. The rearview vision system in claim 22 wherein said atleast three image capture devices are aimed along non-parallelhorizontal axes.
 31. The rearview vision system in claim 22 wherein saidportion which is compressed is a lower vertical portion of said imageportion from said center image capture device.
 32. The rearview visionsystem in claim 23 wherein said image portion from said center imagecapture device is made up of a plurality of horizontal lines and whereinsaid portion of said image portion is compressed by deleting particularones of said horizontal lines.
 33. The rearview vision system in claim25 wherein said visual prompt is an outline of an area which wouldsubstantially be occupied by said vehicle in said displayed image. 34.The rearview vision system in claim 28 wherein said dead space includesperspective lines which are aligned with said direction of travel. 35.The rearview vision system in claim 32 wherein a greater number of saidhorizontal lines are removed further away from a vertical center of saidimage portion from said center image capture device.
 36. The rearviewvision system in claim 32 including adjusting values of adjacent pixelsas a function of pixel values of removed ones of said horizontal lines.37. A rearview vision system for a vehicle having a gear actuator,comprising:at least two image capture devices positioned on the vehicleand directed rearwardly with respect to the direction of forward travelof said vehicle; a display system which displays an image synthesizedfrom outputs of said image capture devices; and said display systemproducing a graphic overlay superimposed on said display image visibleto the driver informing the driver of what is occurring in the areasurrounding the vehicle.
 38. The rearview vision system in claim 37wherein said graphic overlay is in the form of indicia of theanticipated path of travel of the vehicle.
 39. The rearview visionsystem in claim 37 wherein said graphic overlay has a form that is afunction of at least one of the direction or travel and speed of thevehicle.
 40. The rearview vision system in claim 37 wherein said graphicoverlay is a distance indicia indicating distances behind the vehicle ofobjects.
 41. The rearview vision system in claim 37 wherein thedisplayed image includes a visual prompt of the location of said vehiclein said view.
 42. The rearview vision system in claim 37 wherein thedisplayed image approximates a rearward-facing view from a singlelocation.
 43. The rearview vision system in claim 37 wherein saiddisplay system provides an additional enhancement to the displayedimage.
 44. The rearview vision system in claim 40 wherein said indiciahas a form that responds to the rate of turn of the vehicle.
 45. Therearview vision system in claim 40 wherein said distance indiciacomprises at least one horizontal mark superimposed on the displayedimage.
 46. The rearview vision system in claim 41 wherein said visualprompt is an outline of an area which would substantially be occupied bysaid vehicle in said view.
 47. The rearview vision system in claim 41wherein said dead space includes perspective lines which are alignedwith said direction of travel.
 48. The rearview vision system in claim42 wherein said single location is forward of the driver with respect tosaid direction of travel.
 49. The rearview vision system in claim 43wherein said additional enhancement includes visually distinguishingimages of objects too close to said vehicle for safe lane-changemaneuver.
 50. The rearview vision system in claim 44 wherein saidindicia form responds to at least one of the vehicle's steering system,the vehicle's differential system, and a compass.
 51. The rearviewvision system in claim 49 wherein said visually distinguishing imageincludes at least one of displaying the object too close to said vehiclein a particular color and flashing the image of the object too close tosaid vehicle.
 52. A rearview vision system for a vehicle having a gearactuator, comprising;at least two image capture devices positioned onthe vehicle and directed rearwardly with respect to the direction oftravel of said vehicle; a display system which displays an imagesynthesized from outputs of said image capture devices; and imageenhancement means for enhancing the displayed image including a graphicoverlay superimposed on said image wherein said graphic overlay includesindicia of the anticipated path of travel of the vehicle wherein saidgraphic overlay is disabled when the vehicle's gear actuator is not inreverse gear.
 53. The rearview vision system in claim 52 wherein saidindicia has a form that responds to the rate of turn of the vehicle. 54.The rearview vision system in claim 53 wherein said indicia formresponds to at least one of the vehicle's steering system, the vehicle'sdifferential system, and a compass.